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Inside This Brief 



In 2007, Washington State launched an innovative 
performance incentive policy for community 
colleges, called the Student Achievement Initiative 
(SAI). The purpose of the SAI is to use data and 
performance funding to motivate colleges to 
implement systemic changes in practice that lead to 
improved student outcomes. The SAI seeks to 
address some of the key shortcomings of previous 
state higher education performance incentive 
policies and so has attracted a lot of interest from 
policymakers and funders nationally By drawing 
on initial observations from our ongoing evaluation 
of the SAI as well as knowledge about earlier- 
generation performance funding policies attempted 
elsewhere, this brief aims to inform the 
conversation currently going on in many states 
about using state policy levers to meet ambitious 
state and national goals for increased college 
attainment. To that end, this brief calls particular 
attention to a number of policy choices that the 
Washington community and technical college 
system faced as it designed and began to implement 
the SAI across its 34 colleges. These are some of the 
same key choices policymakers and college leaders 
in other states will confront should they begin to 
design and enact performance incentive policies as 
means to improve community college student 
outcomes: 

• Complexity in the measurement framework: 
What is the best way to balance simplicity and 
comprehensiveness in selecting the specific 
measures to use in a performance incentive 
system that rewards intermediate milestones as 
well as completions? 

• Nature of attainment data to he reported: 
Should cross-sectional or longitudinal student 
data be used to compute achievement points? If 
cross-sectional data are used, how can they be 
connected to longitudinal cohort analysis to 
better inform institutional improvement efforts? 

• Defining college performance: Is it more prudent 
to reward colleges on the basis of gains in "total 
points" or on the basis of gains in "points per 
student"? 



• Proportion of performance-based funding: What 
portion of the total institutional budget should 
be distributed on the basis of performance? 

• Source of performance funds: Should 
performance dollars be new appropriations or 
should they be reallocated from institutional 
base budgets? What is a reasonable time period 
and process for reaching this ultimate level of 
performance funding? 

• Mechanism for allocating performance funds: 
Should performance be factored into the basic 
funding formula (regardless of funding level 
and source) or should performance funds be 
allocated as a bonus on top of regular fimding? 

• Buy-in and engagement: What activities, such as 
the creation of a broadly representative task 
force for developing overarching design 
principles or the establishment of a low-stakes 
"learning year" during early implementation, 
are conducive to gaining the support of key 
college persormel, such as college presidents 
and institutional researchers? 

• Technical assistance: What sorts of assistance 
should states provide to help colleges learn how 
to use data to continuously improve programs 
and services? 
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Introduction: 

Washington State's Student Achievement 
Initiative and the National Imperative to 
Improve Community College Completion 



A mid growing signs of America's weakening 
position in the global economy, federal and 
state policymakers and major foundations 
have set ambitious goals for increasing 
postsecondary attainment in the United States. 
Given changing U.S. demographics, it has become 
clear that these national goals are attainable only 
with vastly improved outcomes among traditionally 
less-successful populations, which include 
academically underprepared students who come 
from poor primary and secondary schools, adults 
who are working in low-wage jobs or are 
unemployed, and immigrants with limited English 
proficiency. These populations enroll 
predominantly in the nation's community colleges. 
Accordingly, much of the national mobilization 
around college completion is focused squarely on 
community colleges as well as on state policy, 
because community colleges receive most of their 
funding from state and local sources. 

Given declining public resources for public higher 
education and skyrocketing enrollments, better 
outcomes from community colleges will require 
gains in productivity. Colleges will need to learn 
how to increase student progression and success 
with the same or fewer resources. Doing so will 
require community colleges to make systemic 
changes in their policies and practices. 

It is within this context that the Washington State 
Student Achievement Initiative (SAI) has attracted 
national attention. As states strive to increase 
community college student success, many are 
looking at two policy levers — data and performance 
funding — to motivate and guide colleges to 
implement changes in practice that lead to 
improved student outcomes. Washington's State 
Board for Community and Technical Colleges is at 
the forefront of efforts to develop better measures of 
student success and to use those enhanced 
measures as the basis for rewarding colleges for 
increasing student achievement. In adopting and 



implementing the SAI, the State Board is seeking to 
use data and fiscal incentives to drive systemic 
institutional improvement and thus increase 
productivity across its 34-college system. 

This brief examines key policy issues raised by 
Washington State's experience to date with its 
vanguard performance incentive policy. It draws on 
our observations from the first year of a three-year. 
Gates Foundation-funded evaluation of the SAI as 
well as from our experience with these issues in a 
national context. Because the evaluation of the SAI 
and the SAI policy itself are still in their early 
stages, it is too early to assess the long-term 
effectiveness of this effort. However, insights from 
Washington State's early experience with the policy 
can help inform the conversation currently 
occurring in many states on how to use state policy 
levers to meet ambitious state and national goals for 
increased college completion. To that end, the brief 
examines policy challenges and choices that the 
Washington community and technical college 
system has grappled with as it designed and is now 
implementing the SAI, and that policymakers and 
college leaders in other states will have to address 
in their efforts to improve community college 
outcomes. 
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The Student Achievement Initiative: 
A Next-Generation Performance 
Incentive Policy 



N either performance measurement nor 

performance funding is new in U.S. higher 
education. Both have decades-long histories 
as part of the growing interest in public 
accountability and efficient government.^ But the 
traditional means of measuring and accounting for 
outcomes in community colleges have proven 
inadequate,^ and many of the earlier-generation 
state performance fimding policies have been 
abandoned.^ In developing the Student 
Achievement Initiative, the Washington community 
and technical college system sought to address the 
shortcomings of earlier policies, including previous 
policies enacted in its own state. 

Limitations of Traditional Approaches 
to Measuring and Rewarding Success 

Traditional measures of postsecondary student 
outcomes are typically focused on completion rates 
for students in degree programs. Yet community 
colleges serve many students who start in remedial 
or even adult basic skills programs and who may 
take years to progress toward credentials. Measures 
that focus on completion in a set period of time thus 
put colleges that serve large numbers of 
underprepared students at a disadvantage. As such, 
they might discourage colleges from serving these 
students, which is contrary to the community 
college "open access" philosophy. Moreover, while 
outcome measures such as graduation rates are 
important, they provide no information on where 
students struggle on their paths to earning a 
credential, thus offering colleges little information 
on what they can do to improve their performance. 

Previous attempts to design performance funding 
schemes for postsecondary education have also 
proven inadequate.^ A primary reason is that they 
have typically been devised without much input 
from college educators, who often do not embrace 
the definitions of "performance" reflected in the 
chosen metrics. Systems that reward completion 
alone have been especially unpopular across 



community colleges because they do not reflect the 
full range of missions assigned to the colleges. By 
providing rewards primarily for completion, these 
earlier performance funding systems were feared to 
encourage colleges to turn away from their historic 
mission to serve underprepared students, who are 
the least likely to succeed. Many performance 
funding systems set targets for colleges to meet in 
order to earn rewards, which led to controversies 
about reasonable expectations for open-access 
institutions. Finally, these earlier performance 
funding systems were financially unstable. 
Proposals to carve performance funding from 
college base budgets were met with stiff political 
resistance. But funding systems that relied on 
"new" money often fell victim to budget cuts as 
institutions fought to protect their base budgets at 
the expense of special programs. 

Another weakness of the traditional approaches to 
both performance measurement and performance 
funding is that the impact of these policies did not 
penetrate to the level of classroom faculty and 
student support personnel whose interactions with 
students must change before substantial 
improvements in student learning and completion 
can be expected.^ Thus, these tools have had more 
relevance in the arenas of public accountability and 
political decision-making than in institutional 
improvement and effectiveness. 

Perhaps because of these limitations, there is scant 
evidence, if any, that performance reporting and 
performance funding, separately or in combination, 
have led to any fundamental changes in 
institutional practice or effectiveness.^ 

Development and Design Features 
of the SAI 

Like many states, Washington implemented 
performance funding in the 1990s, but that effort was 
short-lived, lasting only from 1997 to 1999. Its demise 
is attributable to a number of factors, including lack 
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of support from the colleges, which were being 
affected by a policy that was developed and imposed 
by a coalition consisting largely of legislators and 
business leaders/ This time around the impetus 
came from within the system. In conceiving the SAI, 
the State Board was responding to heightened 
expectations from policymakers and the public for 
accountability and improved college outcomes, but 
the Board did this in a proactive way intended to 
reflect the mission and values of the college system 
better than the earlier efforts had done. 

In September of 2007, the State Board for 
Community and Technical Colleges launched the 
Student Achievement Initiative for Washington's 
two-year colleges, following its development by a 
task force comprising State Board members, college 
trustees, presidents, and faculty representatives. 

The policy that emerged from the task force 
reflected a set of design principles that would guide 
the implementation of the SAI (see box, "SAI 
Design Principles").* These principles indeed 
shaped the development of the two main 



components of the policy — the measurement 
framework and the funding mechanism. 

SAI Measurement Framework 

The Student Achievement Initiative sets forth a 
framework for computing the number of 
"achievement points" attained by students enrolled 
in the state's 34 public two-year colleges. Colleges 
earn points when students achieve one or more of 
these educational milestones, which are organized 
along a continuum from remedial programs (which 
include adult basic education and pre-college, 
"developmental" education) up through the 
completion of credentials and training programs. 

An achievement point is generated for attainments 
in any of four areas: 

(1) Building toward college-level skills (two types 
of attainments) 

a. Basic skills gains — increase in skill level 
based on a standardized test (multiple 
points attainable by a student) 



SAI Design Principles 

Overall Principles: 

• The initiative leads to improved educational 
attainment for students, specifically the goal 
of reaching the "tipping point" (of at least a 
year of college and an occupational certificate) 
and beyond. 

• The initiative allows colleges sufficient 
flexibility to improve student achievement 
according to their local needs. 

• The initiative results in the identification and 
implementation of successful practices to 
improve student achievement system-wide. 

Principles for Measurement: 

• Performance measures recognize students in 
all mission areas and reflect the needs of the 
diverse communities served by colleges. 

• Performance measures must measure 
incremental gains in students' educational 
progress irrespective of mission area. 

• Measures are simple, understandable, and 



reliable and represent valid points in students' 
educational progress. 

• Measures focus on student achievement 
improvements that can be influenced by 
colleges. 

Principles for Incentive Funding: 

• Colleges are rewarded for improvements in 
student achievement. 

• Funding is structured so that colleges compete 
against themselves for continuous 
improvement rather than competing with 
each other. 

• Funding is stable and predictable, and 
cumulative over time. 

• Incentive funding rewards student success 
and becomes a resource for adopting and 
expanding practices leading to further 
success. 

• New funds provide the greatest incentive. 
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b. Passing pre-college writing or math 
courses with a grade that would qualify a 
student to advance to the next level 
(multiple points attainable by a student) 

(2) First year retention and progress 

a. Earning 15 quarter credits of college-level 
course work 

b. Earning 30 quarter credits of college-level 
course work 

(3) Completing college-level math (passing math 
courses required for either technical or 
academic associate degrees) 

(4) Completions (degrees, occupational 
certificates, apprenticeship training) 

Notably, the achievement points encompass the full 
range of mission areas, including basic skills, 
remedial, workforce education, and academic 
transfer. The State Board provides the colleges with 
quarterly student unit record data (the colleges 
operate on a quarter term system) to track trends in 
the attainment of achievement points by their 
students. The data are intended to help colleges 
identify opportunities for improvement and 
evaluate the impact of efforts to increase 
achievement. The achievement points are computed 
for each college based on the total achievements by 
all students over the course of one year (with 
quarterly reporting along the way). 

SAI Funding Mechanism 

The measurement framework yields data on total 
achievement points earned by category of 
achievement, but colleges receive fimding for 
increases in total achievement points attained by 
students in a given year compared to the baseline 
year. The baseline year was 2006-07 for the first 
round of performance funding, but it became the 
prior year after that initial round. Eunding per point 
gain is determined by the amount of dollars 
available to fund the initiative and the total point 
gain achieved across the system. The State Board 
estimates the size of the point gain at the start of the 
year and announces the approximate value of a 
point gain that can be earned. Eunding per point 
gain in the first two years of performance funding 
was $31 and $40, respectively. 

The implementation schedule for SAI funding 
awards was designed to help colleges understand 
and adapt to the metrics and funding rules: 



• 2006-07 was established as the base year for 
measuring colleges' initial performance. 

• 2007-08 was designated as the "learning year" 
for colleges to begin to use data and develop 
institutional responses prior to the introduction 
of performance funding; each college received 
$51,000 in seed money (which was added to 
their base budgets). 

• 2008-09 was the second year of seed money 
($66,000 of base budget funding per college). 

• In fall 2009 colleges collectively received $1.8 
million for performance gains in 2008-09, in 
addition to retaining the increased base budget 
funding from the prior years. At a rate of $31 per 
point gain, the average performance-generated 
funding award was $60,000 per district. Of the 
$1.8 million, $800,000 was granted on a one-time 
basis by the Eord Eoundation and was not 
carried forward into college base budgets. 

• In 2010-11 colleges will retain the state-funded 
awards from the prior years and receive an 
additional $1.8 million for performance gains 
achieved in 2009-10, funded at $40 per point 
gain. The Gates Eoundation provided $800,000 
of this amount, which, as a one-time grant, will 
not be carried forward into college base budgets. 

The SAI funding mechanism does not affect the 
formula by which the bulk of the system's budget is 
allocated to colleges. SAI funding is provided as a 
financial reward in addition to the amount each 
college receives through the system's basic funding 
mechanism. Once earned, however, the reward is a 
bonus to be added to the college's base budget. Eor 
now, the SAI funds constitute less than one percent 
of the system's total budget. The intent is to 
gradually increase reward amounts. 

How the SAI Addresses the 
Limitations of Earlier Policies 

The measurement system devised for the SAI 
addresses one of the key concerns with traditional 
metrics by monitoring student progression through 
intermediate milestones on the way to program 
completion, in addition to tracking completions. 

The milestones were chosen on the basis of research 
into where students tend to struggle and drop out.^ 
If students can be helped to reach these key 
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transition points, research suggests they will build 
momentum toward completion. A continuum of 
milestones was developed to span all mission areas 
of the colleges, with all students, both noncredit and 
credit, eligible to earn points as they progress. 
Through another design feature that aimed to 
improve upon traditional accountability for 
outcomes, the State Board gives colleges timely data 
on achievement points earned by students to help 
them identify areas of weakness and opportunities 
for improvement. 

In an effort to increase support for the initiative 
from across the college system, the State Board 
convened a task force of college representatives to 
develop the SAI measurement system, informed by 
the research on progression milestones. The task 
force's decision to define performance measures 
around progression points has resonated with 
policy reform initiatives across the country that are 
seeking to improve college completion rates.^° 

Some of the better known examples are the 
Complete College America initiative, which has 
developed metrics in concert with the National 
Governors Association; the Achieving the Dream 
initiative and its Developmental Education 
Initiative; and the Access to Success initiative of the 
Education Trust and National Association of System 
TTeads. As activity around intermediate measures 
grows, the experience of Washington's SAI will be 
increasingly valuable in helping to establish 
cormections between intermediate attainments and 
progression toward ultimate outcomes. 

The design of the fimding features of the SAI was 
also an attempt to rectify some weaknesses in 
previous performance funding models. By 
rewarding colleges with fimding for helping 
students reach these intermediate points, rather 
than just for completion, a greater portion of college 
efforts are recognized. Concerns about creating 
disincentives for colleges to serve underprepared 
students are addressed in two ways. Eirst, the point 
system emphasizes the progression of students in 
and through pre-college study, effectively according 
great value to colleges' work with underprepared 
students. Second, colleges earn points based on 
their own improvement over time rather than on 
any targets or comparisons with sister colleges. 
Thus, colleges that serve more disadvantaged 
students are not penalized for doing so. 

The underlying premise of the SAI is that the 



reporting of data and the financial rewards for 
student achievement will motivate and guide 
colleges to undertake systemic changes in practice 
to improve student outcomes on a substantial scale. 
These systemic changes are what so many states are 
seeking today as they attempt to raise education 
levels. It is especially noteworthy that, unlike many 
state performance funding systems that were 
imposed from the outside, in this case the State 
Board took the lead in developing the SAI. The 
internal ownership and management of the 
initiative is, then, another way in which the SAI 
attempts to overcome the shortcomings of previous 
policies. 





Designing Community College 
Performance Incentives: 

Policy Challenges and Choices 



T he Washington community and technical 

colleges deserve great credit for introducing a 
performance incentive policy that seeks to 
address the l im itations of earlier policies and that is 
focused centrally on improving student outcomes. 

The State Board identified student success as such an 
important priority for the future of Washington that it 
took the risky step of pioneering a new approach 
across an entire community college system. Beginning 
the design process with a set of principles developed 
by a task force with representation from across the 
system helped immensely in gamering initial buy-in 
and in easing some of the anxiety that necessarily 
accompanies policy innovation. Still, the process 
involved making hard choices about the design of the 
incentive system. 

In this section we describe core policy issues that 
the Washington State two-year college system faced 
in designing the SAI, with a focus on its three 
primary components: 

• Performance measures 
• Performance funding 
• Support for institutional change. 

We discuss how decisions about these issues seem 
to be playing out early on in the implementation of 
the SAI in Washington State, and we present key 
choices for policymakers to consider in developing 
their own performance incentive systems to 
improve college performance while ensuring 
accountability. 

Designing a Valid and Useful 
Measurement System 

The experience to date with the SAI in Washington 
suggests that a performance measurement 
framework based on intermediate measures of 
student progression is viewed by faculty and staff 
as more valid than traditional measurement systems 
and thus can help colleges to focus on student 
success. Most of what are considered leading states 
in addressing performance accountability have now 



begun to incorporate measures of student 
progression. In deciding how to attach points to the 
progression measures, the SAI designers opted for 
simplicity and timeliness of reporting by using 
armual counts of total points earned by all students. 
This approach has raised some challenges for 
colleges in trying to use achievement point data to 
inform their efforts to improve programs and 
services on the ground. 

Washington State's Approach 

Washington's effort to design performance 
measures for the SAI began from a strong 
foundation that used research from the State Board 
research staff and the Community College Research 
Center to help determine the key points where 
students get stuck and the achievements that 
provide momentum for further progress and 
completion. In our interviews with colleges, we 
heard widespread support for this research, which 
helped to lend credibility to the measures. 

Grounding the performance measures in a set of 
design principles helped to avoid some of the pitfalls 
of other measurement systems. For example, the 
commitment to recognize student progression across 
mission areas confronted head-on one of the most 
common complaints about performance 
measurement — that it penalizes colleges that serve 
large numbers of educationally disadvantaged 
students. Similarly, the design principle to measure 
only those achievements that can be influenced by 
colleges sent a strong message that this internal effort 
would avoid the problems encountered by those 
measurement systems designed by outsiders who do 
not understand the challenges that colleges face. 
Among the most important design principles, which 
was indeed followed, is that the measures should be 
simple and understandable. Too many measurement 
systems have failed to deliver because they have 
tried to incorporate too many measures. It is 
especially impressive that a broadly representative 
task force was able to agree on a simple framework. 



8 




Our preliminary field research in Washington State 
indicates that, among those who are familiar with it, 
the achievement point framework is quite popular 
across the colleges. In part, this seems to be a result of 
its ability to provide a common vocabulary around a 
common goal — student progression. It is important 
to recognize that the SAI was not introduced in a 
vacuum. The colleges already had a range of student 
retention and strategic enrollment management 
initiatives underway, and they are finding the SAI 
achievement point framework helpful in giving 
cohesion to a number of sometimes fragmented 
existing initiatives. In this environment of heightened 
public accountability, some college leaders are 
finding the framework valuable for communicating 
with trustees, accreditation agencies, grantors, and 
others about their student success goals and their 
efforts to meet them. The framework also seems to 
have elevated the importance of adult basic skills and 
developmental education. By rewarding 
achievements in those areas, the initiative has 
signaled not only that these are important functions 
of the colleges but also that student success in those 
areas is vital to increased attainment of certificates 
and degrees. 

Key Choices for Policymakers 

While the overall SAI achievement point framework 
has proved to be compelling to college leaders in 
and outside of Washington, three of the specific 
design decisions reflected in the framework raise 
some trade-offs that should be considered carefully 
by those thinking of adopting a performance 
measurement system. 

(1) Complexity in the measurement framework 

First, in seeking simplicity in the measures across 
the full spectrum of missions, the State Board chose 
to focus on a subset of achievement points among 
many points that could have been selected. There is 
always a trade-off between simplicity and 
comprehensiveness in constructing a performance 
measurement framework, and it is not surprising 
that arguments have been presented for more, or 
different, achievements to be counted. In our 
interviews with college persormel, numerous 
suggestions surfaced for "tweaking" the point 
system. Since funding is involved, it is not 
surprising that these concerns tend to reflect the 
program area and job duties of the person 



expressing them. For example, some English faculty 
question why a point is generated for passing 
college math but not for passing college English, 
despite the fact that both are required for an 
associate degree. Similarly, some faculty in 
academic transfer programs question why no points 
are generated for a transfer, since many students 
transfer without earning an associate degree, 
generating no points despite the significant 
achievement. Many such suggestions were offered, 
indicating that people are potentially responsive to 
the incentives embedded in the point system and 
illustrating the trade-offs inherent in the State 
Board's decision to design a simple measurement 
system. Had the Board developed a framework that 
included the additional measures suggested to us, 
we might have heard complaints that the system 
was too complicated. 

(2) Nature of attainment data to be reported 

Second, the achievement points are computed for 
each college based on the total achievements by all 
students over the course of one year (with quarterly 
reporting along the way). This cross-sectional 
reporting of achievements in a given time period is 
quite different from tracking student progress over 
time. For example, tracking student progress over 
time — often called "cohort tracking" — would allow 
reporting of how many students complete 
developmental education within one year, or how 
many complete college math within two years. The 
method chosen for the SAI more simply reports 
how many of a college's students earn a particular 
achievement point in a given year, regardless of 
how long they have been enrolled. Collecting data 
on rates of progression of student cohorts over time 
is more complicated and would involve a lag time 
that the designers wanted to avoid. In addition, the 
SAI designers wanted a simple way to show 
incremental student gains made each year — 
information that is meaningful to policymakers but 
less useful to the colleges themselves as they work 
to identify timely student progression and to 
evaluate the impact of various interventions. 

At the same time, the State Board has recognized 
that SAI data are most valuable when used in 
combination with local student-level transcript 
data. Staff are now working with institutional 
researchers at the colleges to understand how best 
to use SAI data together with local data to generate 
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information useful for improving programs and 
services. These efforts have the potential to promote 
sharing among colleges of effective practices — 
something that could not be done with SAI data 
alone. Colleges ultimately need to conduct cohort 
tracking to identify strategies for improving student 
progression and evaluating and further improving 
those efforts. For example, if a college implemented 
a new college- wide approach to intake and 
orientation, it could use local data to generate a new 
cohort that experienced that innovation and prior 
cohorts that had not experienced it. It could then 
use the SAI framework to compare outcomes across 
cohorts. That is, the rate at which the new cohort 
earned college readiness and first-year progression 
points could be compared to the rate for previous 
cohorts. The SAI measures are valuable when used 
as high-level indicators to see where, along 
pathways toward completion, students are most 
likely to drop out, and to set priorities accordingly. 
Student-level cohort data can then take the analysis 
to a deeper level as the progress of subgroups of 
students can be analyzed with respect to existing or 
new policies and interventions. 

(3) Defining college performance 

Third, again in the interest of keeping things simple, 
the Board chose gains in total points as the common 
denominator to use in distributing performances 
bonuses. In effect, this becomes the "bottom line" 
definition of performance. However, a measure 
based on total points is highly correlated with 
college size (i.e., enrollment). A college may achieve 
point gains by recruiting more students or 
otherwise experiencing an enrollment increase, 
while taking no particular actions to help students 
make better progress. Indeed, a college could 
generate a point gain even if, on average, students 
earned fewer points than in the prior year, if the 
enrollment gain were large enough to outweigh the 
loss in average points per student. Our analysis of 
the points generated by colleges during the initial 
period for which the first performance fimding was 
awarded under the SAI (2006-07 through 2008-09) 
found that about one-fifth of the gain in points was 
attributable to enrollment increases and about four- 
fifths to gains in points per student. 

We believe that if a measurement system is to 
encourage greater student progress (and overall 
system productivity), it helps to use a measure that 



gauges outcomes with respect to inputs. This could be 
some version of points per student or points per 
dollar of expenditure. We favor the former because 
it would inspire needed focus on student progress. 
Tracking absolute points may be consistent with 
state and national goals to increase the volume of 
college degrees, but in a period in which colleges 
will be compelled to increase the number of 
graduates with fixed or declining resources, 
tracking points per student or points per dollar may 
make more sense. 

Designing an Effective 
Performance Funding System 

Performance fimding is a complex topic with 
interconnected technical and political factors 
involved. The past history of performance fimding 
has been a checkered one at best, with most state 
policies proving unsustainable.^^ It is fair to describe 
performance funding as an unsettled paradigm with 
a lack of consensus on effective design principles. 

We noted earlier that Washington's SAI has 
addressed some of the serious limitations of early 
generations of performance funding by developing 
a definition of performance that encompasses all 
mission areas and avoids penalizing colleges that 
serve underprepared students. But far less settled is 
the question of how to reward performance 
financially, once performance is defined. We find it 
helpful to distinguish among three features of any 
funding system that attempts to create fiscal 
incentives for performance, that is, three choices 
that policymakers face: 

(1) The amount of performance funds: What portion 
of the total institutional budget is distributed 
on the basis of performance? 

(2) The source of performance funds: Are the 
performance dollars new appropriations or are 
they reallocated from institutional base 
budgets, or some combination? 

(3) The mechanism for allocating performance funds: 
Is performance factored into the basic funding 
formula (regardless of funding level and 
source) or are performance funds allocated as 
a bonus on top of regular funding? 

We have observed that the second and third choices 
are often conflated in discussions about the design 
of performance funding. It is often assumed that if 
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new funds are identified to support performance 
they somehow need to be set aside and used to 
reward performance after base institutional funds 
are distributed on the basis of enrollment. But new 
funds could be added to the base with the 
augmented total distributed by means of a formula 
that rewards enrollment and performance in some 
combination. Conversely, a percentage of the base 
budget could be set aside to be used as bonus funds 
to reward performance without changing the basic 
enrollment-driven funding model for the balance of 
each institution's budget. The point is that there are 
choices to be made on all three dimensions of 
performance funding design. 

Washington State's Approach 

In designing the funding mechanism, the State 
Board and the SAI task force drew upon expert 
opinion about performance funding and made a set 
of considered choices. Based on this advice, the 
State Board and the task force designed a 
performance funding approach that (1) at least 
initially devotes less than one percent of the system 
budget to performance with no clear plans to phase 
in increases; (2) relies, in principle, on new funding 
alone, and (3) leaves the base funding formula 
unchanged while allocating SAI performance 
dollars as bonuses. To mitigate opposition to the 
new policy, Washington started small. The "learning 
year," in which colleges were given non-competitive 
funding to lay the groundwork locally, and the 
small budgetary stakes thereafter have reduced 
resistance up to this point. The State Board has also 
stressed that performance funding is for purposes 
of internal improvement and not for punishment or 
inappropriate comparisons. 

Key Choices for Policymakers 

Washington State has so far gone through two 
cycles of SAI performance funding. Based on this 
experience, and in light of a dramatically changed 
fiscal situation in Washington and most other states, 
we can examine in hindsight crucial issues that flow 
from initial design decisions. 

(1) The amount of performance funds 

Definitive evidence on how much performance 
funding it takes to stimulate systemic change is 
lacking (because most past attempts have dedicated 



only small amounts to performance). However, the 
consensus among the college leaders we 
interviewed in Washington is that the amount 
allocated so far through the SAI — less than one 
percent of the budget — is insufficient to inspire the 
kinds of fundamental systemic changes the State 
Board is seeking. To date, colleges seem to have 
spent the modest funding they received from the 
SAI on relatively small-scale, isolated activities. 
Some states that have borrowed heavily from the 
design of the SAI measurement framework have, in 
fact, chosen different approaches to funding.^^ In 
contrast to Washington, Ohio has set 20% as the 
budget target for full implementation of its 
community college performance funding policy, 
which is based in part on the Washington 
achievement point model, and other states have 
similarly set their sights on a significant portion of 
budgets dedicated to funding performance. Still, 
regardless of the portion decided on, we believe 
that it is prudent to phase in performance funding 
gradually so that colleges have time to learn from 
the data and implement better practices. A slow, 
staged implementation toward a significant level of 
funding supports the goal of continuous 
improvement. While policymakers may push for 
quicker implementation for public accountability 
purposes, such approaches are understandably 
viewed as punitive and counterproductive from 
inside the academy. 

(2) The source of performance funds 

In Washington, the State Board was advised by 
outside experts at the outset that using new funds is 
more effective in creating fiscal incentives to increase 
student success. Certainly new funds are more 
politically popular, as no college would choose to 
lose base funds over earning new funds. But 
whether new funds are more effective than 
reallocated funds in changing behavior is an 
unresolved empirical question. Regardless, the 
design principle to rely on new funds became 
impossible to follow as the budget situation 
deteriorated in Washington State. As the 
community college system budget was being 
reduced significantly over the last two years, the 
SAI was funded increasingly through base 
reallocation. The popularity of the SAI with the 
governor and the legislature very likely inoculated 
the system against even larger budget cuts, as 
lawmakers agreed to preserve some funding that 
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might otherwise have been cut as long as it was 
used for performance. Yet the college leaders we 
interviewed did not seem to have considered the 
political capital that the SAI may have bought them; 
instead they focused on what they perceive as a 
reneging on the promise not to take away funds to 
support the SAI. One lesson emerging from this 
difficult situation may be that more research is 
needed to examine the impact of new base budget 
incentives on institutional behaviors. What is clear, 
however, given the fiscal realities facing public 
postsecondary education, is that states 
implementing new performance funding programs 
over the foreseeable future should not structure 
them around new funding if they have any 
intention of devoting a significant amount of funds 
to performance. 

(3) The mechanism for allocating performance funds 

As with the prior choice of new versus reallocated 
funds, this choice necessarily confronts a powerful 
institutional point of view that colleges are already 
underfimded for basic enrollment and operations. 
According to this view, major new initiatives should 
be accompanied by specially designated funds. 
Colleges see increasing numbers of what they call 
"unfunded enrollments" and believe they can 
justify base budget increases on those grounds 
alone. If they are to be expected to make 
fundamental performance gains, they expect special 
funding designated for that purpose. This attitude 
has become pervasive across American higher 
education as lawmakers repeatedly add categorical 
programs or other budget line items to accomplish 
policy objectives. 

The bonus mechanism adopted by the State Board 
in Washington reflects this perspective by leaving 
the base budget allocation methodology unchanged 
and creating a pool of additional bonus funding. Yet 
even in the best of budgetary times, this 
arrangement could be seen as carrying the message 
that colleges are funded to operate, irrespective of 
performance levels, and are expected to improve 
their performance only to the extent that special 
funding becomes and remains available. In the 
worst of budgetary times, as have hit Washington 
and other states, the bonus mechanism becomes 
very difficult to explain. We believe that it is not 
credible to tell colleges they are getting a bonus 
forperformance when, from their perspective, they 



have had their budgets cut by far more than the 
amount of the bonus. Several college leaders we 
interviewed were upset by what they see as the 
"raw deal" of having to earn back even just a 
portion of the funds they have lost. Although not 
materially different in terms of total resources, it 
seems that a more credible message would be that 
today's fiscal realities require the state to invest its 
scarce resources in success. That, in turn, would 
require a phased-in change to the way colleges are 
funded — moving away from straight enrollment 
funding toward funding colleges in part for their 
ongoing work to help students make progress and 
complete their education. Colleges might still 
complain about having to earn back their lost funds, 
but the reason for having to do so would be clear: 
they are living in a new fiscal reality in which 
taxpayers invest in performance. 

The most important lesson from the performance 
funding aspect of the SAI implementation to date is 
that communication is critical. Ways must be found 
to simplify explanations of what is by nature a very 
complex undertaking. Despite the vigorous 
communication efforts of the State Board staff, we 
were told at every campus we studied that most 
faculty and staff, including high-level 
administrators, have no familiarity with the 
complexities inherent in performance funding, 
including the design choices noted above and the 
means available for defining "performance" to 
avoid perverse incentives, such as discouraging 
enrollment of underprepared students. States will 
be well served by engaging college leaders in 
ongoing discussions about the goals, choices, and 
strategies attendant to performance funding in their 
states. Protecting funding levels in Washington and 
elsewhere may increasingly require college systems 
to incorporate performance incentives into their 
base funding formulas. As state policymakers work 
to invest state funds most productively to increase 
college attainment and improve economic 
competitiveness, current methods of funding 
colleges may no longer be politically and 
economically sustainable. Common ground among 
policymakers and colleges might be found in the 
claim that performance funding can enhance total 
funding which, in turn, allows college faculty and 
staff to focus more intently on student success. 
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Fostering Conditions for 
Institutional Change 

We have already described what will be required for 
the data to become more useful for identifying 
effective practices and for the funding incentives to be 
more likely to motivate systemic change. Research 
and reform around the country have shown that data 
and incentives alone may not be sufficient, however. 
Colleges can collect and report data as a compliance 
activity without learning anything or doing anything 
as a result.^^ New fiscal incentives often fail to 
penetrate the collective consciousness of institutions, 
and even if they do, they can lead to gaming or other 
accounting devices that only give the appearance of 
improvement toward the intended outcomes. Here 
we consider the conditions that promote systemic 
change and how states can foster them. 

Washington State's Approach 

The State Board's strategy for achieving institutional 
change has primarily involved gaining internal buy- 
in and engaging in extensive educational sessions 
about the SAI across the system, particularly during 
the early stages of planning and implementation. 
Acting on the knowledge that performance reporting 
and performance funding policies imposed from the 
outside have not worked well and have not gained 
buy-in from the institutions they affect, the State 
Board formed a task force with broad representation 
from across the system to develop the SAL College 
presidents were a critical part of the task force, as on- 
the-ground leadership is vital to instilling a culture 
supportive of continuous improvement. That has 
been an invaluable factor in the sustainability of the 
SAI to this point. Also valuable was the learning year 
that helped colleges understand the new initiative in 
a low-stakes environment. Throughout that year the 
State Board engaged in a major communication effort, 
with extensive site visits and televised sessions 
broadcast to various segments of the college 
communities. In the subsequent years, because of 
resource constraints and the need to address other 
priorities, the Board reduced the scope of its 
communication efforts but attempted to keep the 
colleges informed through the system's presidents' 
council and through its work with the various 
statewide professional councils that have 
representatives from every college. Board staff also 
worked extensively with college institutional 
researchers to address identified problems with the 



data, knowing that the validity of the data is a 
precondition for the success of the initiative. Recently, 
the State Board began to work with the colleges to 
provide much more hands-on technical assistance on 
using data for improvement. There is a good 
infrastructure in place across the system that allows 
the Board to do this on a system-wide basis. Working 
with the various professional coimcils (e.g., basic 
skills, workforce, and academic transfer) and 
including college institutional researchers in these 
meetings, the Board can greatly improve the 
prospects for data-informed improvement across the 
system. 

Key Choices for Policymakers 

Achieving a change in institutional culture in any 
organization is difficult, and doing so demands a 
great deal from institutional leaders. But in higher 
education, with shared governance and shared 
leadership, the change process is especially 
challenging. 

(1) Buy-in and engagement 

From our interviews we identified presidents and 
institutional researchers as key players influencing a 
college's response to the SAI — the former for 
obvious reasons and the latter because researchers' 
attitudes about the validity and usefulness of data 
can promote or impede willingness to use the data 
as a basis for decision-making. For their part, 
presidents face some strategic choices. For example, 
do they frame the SAI as a discrete initiative and 
work with faculty leaders and others to implement 
it? Or does that risk a backlash of "just another 
thing we have to add to our list when we're already 
over-worked"? Is it more effective to frame the 
initiative as a way to bring together existing college 
initiatives and establish a common vocabulary 
around them? Presidents also face a choice about 
how quickly and thoroughly to engage the 
campus — particularly the faculty — around the SAI. 
Given a history of faculty opposition to 
performance fimding and a history of many short- 
lived reforms, is it best to take a more muted 
approach at the outset, perhaps until it is 
abundantly clear to faculty that the initiative is not 
going away? 

Another important choice to be made by presidents 
involves strategies for using the performance funds 
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earned by their colleges. In Washington, some 
presidents have used the funds to support small- 
scale efforts aimed at improving student success, 
such as a new orientation program for special 
populations or additional advising staff. Other 
presidents have so far chosen to fold SAI funding 
into the overall college budget. The former 
investments, while garnering support among the 
direct recipients, were too small to have a systemic 
impact while the latter approach limited awareness 
of the fimds that are intended to spur new 
institutional behaviors. 

Research on practices associated with 
organizational improvement in and outside of 
higher education suggests that deep engagement of 
faculty and staff with student data is another 
precondition of fimdamental change in institutional 
policies and practices.^^ Achieving this kind of 
engagement requires political and technical 
strategies. Politically, it is important to get 
influential faculty on board as spokespersons for the 
value of these kinds of data. It also requires a 
substantial commitment of faculty and staff time, 
including the time of institutional researchers. 

(2) Technical assistance 

Washington's experience suggests that technical 
assistance to colleges is both critical and time- 
consuming. Moreover, providing technical 
assistance to colleges on using data to continuously 
improve programs and services is a new role for 
many state higher education agencies. At the very 
least, state agencies should provide forums where 
faculty and staff can get together from across 
colleges and discuss key problems with student 
progression and share promising solutions, backed 
up with evidence. In Washington, the State Board 
has partnered effectively with the various statewide 
councils of college personnel, in a number of cases 
bringing different councils together to share 
promising practices. Many other states have similar 
statewide organizations and could seek to work 
with them strategically as the Washington State 
Board has done. 

In a period in which community colleges, not richly 
funded to begin with, are facing draconian budget 
cuts even as enrollments increase, any discussion of 
changing the measures and means by which 
colleges are funded is going to be politically 



sensitive. Faculty and staff are likely to argue that 
such changes should wait until college budgets 
return to more normal levels. Yet it is precisely 
because the current climate has likely become the 
"new normal" that states such as Washington are 
charting new ground in seeking to use scarce 
resources in ways that provide incentives for 
colleges to improve outcomes. Commimication is 
essential in this uncertain and challenging 
environment. Fear and resistance are natural urges, 
and a lack of information increases these tendencies. 
State agencies must explain clearly the goals and 
strategies of performance incentive reforms to 
college leaders who, in turn, need to communicate 
these issues across their campuses. Systemic change 
is unlikely to take hold unless the majority of 
stakeholders understand the political and economic 
realities that are shaping strategies around 
performance reporting and funding. 
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Next Steps in the National Conversation 



T he Student Achievement Initiative is a vitally 
important effort from which there is, and will 
continue to be, much to learn. The 
Washington State Community and Technical 
College System deserves accolades for its 
willingness to innovate to such an extent. The 
stature of community colleges and the rates of 
student success across the country would likely be 
higher if all state community college systems shared 
the spirit, commitment, and tenacity of the 
Washington system. 

Evaluation of the SAI continues through 2012. Even 
as Washington considers refinements to its own 
model, the national conversation around 
intermediate measures, performance funding, and 
institutional change can benefit from Washington's 
experience to date in implementing the SAI. In 



addition to considering the policy challenges and 
choices we presented above, we hope that those 
involved in similar efforts will put some thought 
into additional questions to further advance 
collective knowledge about these issues (see box, 
"Performance Incentive Policies: Purther Questions 
for Consideration"). 

National efforts to increase college completion and 
use resources more productively stand to be 
enriched by the Washington experience. Through its 
integrated application of new approaches to data 
and funding, the Student Achievement Initiative is 
raising important questions, yielding some useful 
answers, giving states a model to adapt to their own 
circumstances, and generally providing hope for 
better student outcomes in the nation's precious 
community college sector. 



Performance Incentive Policies: Further Questions for Consideration 



Questions About Performance Measurement 
Systems: 

• How can annual count data on student 
achievement, such as data from the SAI 
achievement point system, be used most 
effectively in combination with student 
longitudinal cohort data as a basis for 
identifying problems and improving 
practices? 

• Given the value of cohort data but the delays 
attendant to tracking progress over time, 
how can colleges get timely feedback on the 
impact of their actions to help students? 

• Does public accountability for intermediate 
student achievements, short of completion, 
help or hurt community college efforts to 
convey their value to external stakeholders 
(general public, business, trustees, etc.) who 
may place more value on the award of 
credentials? 

Questions About Performance Funding Systems: 

• What kinds of messaging and 
communication strategies can increase 
understanding across colleges and among 



policymakers about the purposes of 
performance funding and the design choices 
involved? 

• How can performance funding systems best 
provide incentives for colleges to serve 
underprepared students? 

• How does a college's attitude toward 
performance funding change with experience 
over time, and how does it vary with respect 
to its relative performance? 

Questions About Promoting Systemic 
Institutional Change: 

• What campus practices seem most conducive 
to fostering widespread engagement in data 
for purposes of learning how to improve 
student outcomes? 

• What good models exist for integrating 
performance data into a college's strategic 
planning and institutional improvement 
processes? 

• How can colleges best innovate at scale in 
order to have the best chance of achieving 
systemic change? 
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Endnotes 



1. See Dougherty, Natow, Hare, & Vega (2010); 
Wellman (2002); and Zumeta (2001). 

2. Offenstein & Shulock (2009). 

3. See Burke & Associates (2005) and Dougherty & 
Reid (2007). 

4. See Burke & Associates (2005) and Dougherty et 
al. (2010). 

5. Burke & Associates (2002). 

6. Dougherty & Hong (2006). 

7. Dougherty & Natow (2009). 

8. Washington State Board for Community and 
Technical Colleges (2007). 

9. Leinbach & Jenkins (2008). 

10. Offenstein & Shulock (2010). 

11. See Burke & Associates (2005) and Dougherty & 
Hong (2006). 



12. See, for example, Tennessee Higher Education 
Commission (2010), Ohio Board of Regents 
(2010), and Indiana Commission for Higher 
Education (n.d.). 

13. A national survey of community college 
institutional research practices found that top 
administrators generally do not use data on 
student outcomes for decision making. This is the 
case even in some colleges that have well-staffed 
research departments and strong information 
systems. See Merest & Jenkins (2007). 

14. See Burke & Associates (2005) and Dougherty & 
Hong (2006). 

15. Eor a review of research on ways community 
colleges can improve organizational 
performance, see Jenkins (2011). 
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